E2E: per-test store isolation with automatic teardown#7279
E2E: per-test store isolation with automatic teardown#7279phyllis-sy-wu wants to merge 2 commits intopsyw-0413-E2E-temporary-time-limit-bumpfrom
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
a47982f to
d6a4e19
Compare
d6a4e19 to
9659cf3
Compare
94f4908 to
d872975
Compare
9659cf3 to
2bb6ce4
Compare
e5289d3 to
5c0d260
Compare
77e5ea0 to
5765d59
Compare
5765d59 to
f45019e
Compare
5c0d260 to
0e615ef
Compare
c7ad615 to
75f05bf
Compare
f45019e to
16b2f4d
Compare
75f05bf to
dbcae8d
Compare
16b2f4d to
e673884
Compare
dbcae8d to
898e174
Compare
dffff76 to
6ee7257
Compare
898e174 to
fd1b216
Compare
6ee7257 to
37cde2e
Compare
5786612 to
cff5059
Compare
37cde2e to
6ab5380
Compare
| const storeSlug = ctx.storeFqdn.replace('.myshopify.com', '') | ||
| e2eSection(wCtx, `Teardown: store ${ctx.storeFqdn}`) | ||
|
|
||
| // Phase 1: Uninstall app from store |
There was a problem hiding this comment.
If we are deleting the store in the next step, why uninstalling the app?
There was a problem hiding this comment.
Because deleting a store won't automatically uninstall the apps that were installed on it. Right now, if we delete a store with apps installed, the install count will go stale on dev dashboard, and those apps will not be able to be deleted after the store is deleted! It's a bug I discovered recently, documented here with more details.
|
|
||
| // Navigate directly to the store creation form on admin.shopify.com | ||
| const email = ctx.email ?? process.env.E2E_ACCOUNT_EMAIL | ||
| await browserPage.goto(`https://admin.shopify.com/store-create/organization/${orgId}`, { |
There was a problem hiding this comment.
We will need to switch soon to a CLI command: #7332
6ab5380 to
887eb1e
Compare
cff5059 to
f48eb8b
Compare
f48eb8b to
f13e969
Compare
887eb1e to
ec5e775
Compare
Differences in type declarationsWe detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:
New type declarationsWe found no new type declarations in this PR Existing type declarationspackages/cli-kit/dist/public/node/upgrade.d.ts@@ -1,5 +1,3 @@
-import { getAutoUpgradeEnabled, setAutoUpgradeEnabled } from '../../private/node/conf-store.js';
-export { getAutoUpgradeEnabled, setAutoUpgradeEnabled };
/**
* Utility function for generating an install command for the user to run
* to install an updated version of Shopify CLI.
|

Addressed #7269
WHY are these changes introduced?
E2E tests currently share a single pre-existing dev store configured via
E2E_STORE_FQDN. This causes:teardownAppsilently skips steps, doesn't verify success, and a single failure blocks later cleanupThis PR also lays the foundation for parallel workers (future PR), where shared stores would be impossible since multiple tests run simultaneously.
WHAT is this pull request doing?
Per-test store isolation:
app dev,hot reload,multi-config dev,toml-config dev) create a fresh store via browser automation onadmin.shopify.com/store-create/organization/{orgId}Robust teardown with escalating retry (
setup/teardown.ts):Three independent phases — failure in one never blocks the next:
Phase 1: Uninstall app from store (3 attempts)
/settings/apps, finds app by name, clicks uninstallh2:has-text("Dev Console")→button[aria-label="hide"]Phase 2: Delete store (3 attempts)
/settings/appsto check stateaccess_accountredirect)isStoreAppsEmpty(reloads once if stale)/settings/plan, clicks delete, checkbox, confirmaccess_accountURLPhase 3: Delete app from dashboard (3 attempts, always runs)
button.critical[type="submit"]refreshIfPageErrordetects error pages and refreshesServer error handling (
setup/browser.ts):refreshIfPageError(page)detects 500/502 error pages and auto-refreshesTagged logging (
setup/env.ts):createLogger(tag)produces[e2e][w0][browser],[e2e][w0][cli]formate2eSection()prints section headers:----- SETUP: store ... -----,----- TEARDOWN: app ... -----Exported building blocks (
setup/store.ts):uninstallAppFromStore(page, storeSlug, appName)→ booleandeleteStore(page, storeSlug)→ booleanisStoreAppsEmpty(page)→ boolean (pure page checker, caller navigates)dismissDevConsole(page)→ voidFiles changed:
setup/store.tssetup/teardown.tsteardownAll()orchestrator with 3-phase teardownsetup/app.tsfindAppOnDevDashboard()(with pagination),deleteAppFromDevDashboard()setup/env.tscreateLogger(),e2eSection(),workerIndexsetup/cli.tssetup/browser.tsrefreshIfPageError()teardownAll, store-dependent tests usestoreTestFixtureHow to test your changes?
Checklist
pnpm changeset add